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ABSTRACT 



The development of a multivariate prediction system 
ained at having useful predictors available early in the training of 
potential Arr.y aviators is discussed* Using this system, supervisors 
will be able to relate a predictor score to a probability table, thus 
enabling administrators to make early decisions involving further 
training of Army aviators* (Author/DB) 
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In this paper the development of a multivariate prediction system aimed at having 
useful predictors available early in the training of potential Arr../ aviators is 
discussed. Using this system, supervisors will be aMo to relate a predictor 
score to a probability table, thus enabling administrators to make early decisions 
involving further training of Army aviators* 
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Prefatory Note 



This paper contains a m. mmary of portions of the 
research begun under Exploratory Research 70, Longi- 
tudinal- Analysis cf Aviator Performance, and continued 
under Work Unit PREDICT, Predicting Aviator Success in 
Training and Operational Assignments. The research was 
conducted by rhe Human Resources Reseirch Organization, 
Division No. 6 (Aviation), Fort Rucker, Alabama. 



PREDICTION OF ARMY AVIATOR PERFORMANCE: 
DESCRIPTION OF A DEVELOPING SYSTEM 



Wiley R. Boyles and 
James L. Wahlberg 



For the past several years, a portion of the work effort of HuinRRO Division No. 6 
(Aviation) at Fort Rucker, Alabama, has been devoted to the development of a data bank 
containing information on the characteristics of Army aviators and descriptions of their 
performance. 

Preliminary phases of various aspects of this effc/t have been described in earlier 
papers. In 1967, Prophet reported on work toward developing a noncognitive predictor of 
performance based on reactions to threat of physical harm (1). In 1968, Boyles described 
further .stages in the development of that measure (2). Prunkl and Boyles (3) reported on 
an aspect of the work devoted to development of a combat criterion, and Boyd and 
Boyles (4) discussed the use of attitudinal items from questionnaires as predictors of 
aviator retention in service. 

The purpose of this paper will be to describe the broad framework in which these 
varied researches are being done and to discuss some of the more recent results. First, we 
are operating in the context of a very expensive training program— expensive, and very 
complex in nature. The knowledges and psychomotor skills being imparted to the 
students are complicated technical ones and involve behaviors that are very hard to 
predict or contro'. Because of the complexity of these behaviors, knowledge o? the values 
of many component variables is necessary for good description. We attempt to predict 
these behaviors in a situation in which failure to predict or control with maximum 
efficiency results in a Iremendoi^ amount of money wasted in a very short time. The 
Army has been training about 6u0‘ new aviators per month in a program that requires 
about one year to complete. A rough hut conservative estimate of the cost per student is 
$40,000— a minimum of a 24*million dollar per month training investment. 

One advantage of the situation, from the measurement point of view, is that it 
furnishes a large number of subjects in a short period of time. This affords an excellent 
opportunity to examine a large number of variables simultaneously, and anything 
approaching an adequate description of the events occurring in this type of program 
requires that kind of examination. 

A complication, however, stems from the fact that data require a year to mature 
when the criterion is successful completion of training, and over two years to mature 
when the criterion is job performance. The job is a year’s combat performance in 
Vietnam for virtually all graduates. On site data collection for this job has obvious 
drawbacks, one of the greatest of these being the necessity of non interference in 
behavior that is often a life or death matter. So, job performance data are being gatheicl 
as soon as possible after completion of the job. 



lf I7ie number of aviators being trained has been reduced since the time of this research because of 
changing military requirements. 
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Let me briefly describe the sequence of selection, training, and job performance and 
the data we are collecting (o» have collected) at various stages: 

For all potential aviators, the first step coward the training program is successful 
performance on the primary selection instrument, the Flight Aptitude Selection Test 
Battery {FAST), developed by the Army’s Behavior and Systems Research Laboratory. It 
contains a number of tests which attempt to tap several aspects of cognitive aptitude for 
aviation and some pertinent biographical information. The psychometric characteristics of 
the test battery as a primary selection instrument have been described by Kaplan (5). 

In the PREDICT research, Flight Aptitude Selection Test scores are being deposited 
in the data bank. Previously reported research (6) has shown that these scores are good 
predictors of whether students will fail to complete the program because of poor Hying 
performance. 

The Navy has for several years used a multivariate prediction system in secondary 
selection and their system is built around the use of similar battery (7). This, of course, 
is in spite of the attenuation of prediction resulting from use of the batteries as primary 
selectors in both services. 

The applicant who equals or surpasses the FAST cut-off score and meets the several 
other requirements for entrance into the program goes directly to the primary flight 
school if he has already competed Basic Combat Training (BCT). For the last several 
years, however, about half the input to Army flight training has consisted of young men 
coming into the program from civilian lif". These men must successfully complete BCT 
before entering the flight program proper. 

The 8-week BCT program involves, among other things, physical training, marksman- 
ship training, and introductions to military discipline and tactics. Using the evaluations 
made of the trainee throughout this period, we have collected date, on more than 50 
variables for each of 1,500 BCT students. 

Some preliminary analyses of these data have been completed and have shown 
promise for predicting a troublesome behavior called “Snobird” attrition—attrition from 
the training program, o'ten at the trainee’s request, occurring after the completion of 
BCT but prior to entrance into “preflight” training. In a high volume program such as 
Army aviation training, unpredictable fluctuations in weather (scheduling the start and 
finish dates for training depends on the weather and is necessarily less than precise), 
supply of recruits, and demand for graduates often resuit in over-supplies of personnel at 
various points during training. The trainee who ai rives at the primary training center and 
has a delay facing him before admission to preflight training is termed a “Snobird.” The 
Snobird population at one of the centers has sometimes been e large as 600 and the 
length of the delay is sometimes several weeks. Resignations and other personnel losses 
occur with disturbingly high frequencies during peak Snobird periods, and identification 
of high-probability eliminees prior to entry into this status has potential for reducing 
transportation and training costs. 

Prunkl and Tomolonis are conducting the PREDICT study of the 50 BCT variables, 
and their preliminary results show, for example, a point-biserial correlation of .21 
between scores on a physical proficiency test administered early in BCT and subsequent 
Snobird attrition. This relationship is significant at the .01 level. 2 The comprehensive 
evaluation of their predictors, both against Snobird attrition and against flight training 
criteria, will be completed at a later date. 

Once a trainee has passed the BCT and Snobird hurdles, he enters preflight training, 
w.ie.e quantitative data generated include academic examination scores in military 
development subjects and in aviation subjects. There are two primary classification of 
student aviators in the Army: one is the warrant officer candidate (WOC), who is an 

2 Prunkl, Peter R., and Tomolonis, Peter C.: Personal Communication. 



enlisted man during training and becomes an aviation warrant officer upon graduation; 
the other is the officer student, who has received a commission or warrant prior to 
entering the aviation program. Although both are awarded Army aviator ratings at 
graduation, the training for the two types of students is slightly different. There are, 
therefore, slightly different sets of information available for the two types, and this has 
implications for the design of the PREDICT research. 

After a preflight period of four weeks for warrant officer candidates and approxn 
mately two weeks for officer students, actual flying training begins, the first, or primary, 
portion of which is 16 weeks long. During this period, each student receives about 50 
grades on flying ability. The warrant officer candidate accumulates 27 academic grades 
and the officer student 21 during primary training. All of these grades are placed in the 
HumRRO data bank. Upon completion of primary training, the student attends advanced 
training at Fort Rucker for an additional 16 weeks. The grades for each flight and for 
each academic examination administered there are inc'uded in the data bank. Graduation 
comes at the end of that period. 

Consider now the task of the administrator in this program. He may be called upon 
at any point during the student’s training period to decide whether a marginal student 
should be: 

(1) Retained in the program and allowed to continue with his original class. 

(2) Retained, but given extra training to correct his academic, flight or military 
deficiencies. 

(3) Eliminated from the program. 

Eliminating a student who could complete the program is costly because a substam 
tial investment will be wasted and a replacement must be trained at further cost. 
Retaining a student who is highly likely to fail to complete the program is also wasteful. 
The administrator, therefore, must make accurate decisions as early as possible. Typically, 
however, he is faced with much more data on the individual than he can possibiy absorb 
and integrate into his decision in the short time allotted him. The result is inevitably 
selection of portions of the data on the basis of personal experience, which would vary 
greatly from one administrator to the next. Comparative and cumulative validities of 
variables available as potential predictors are generally unknown. 

The PREDICT project is in the process of developing a multivariate prediction 
system which can use all the daU available on the mtaginal student, up to the point in 
training at which a decision must be made as to his retention or elimination. From the 
mass of data, the system will select the most valid predictors, properly weight them (i.e., 
on an empirical basis), and furnish tne administrator with a predictor score which he can 
relate to a probability table. An example of such a table wiH be given later in the paper. 

Another important decision chat must be made by aviation training administrators 
involves selection for specialized training. An Lmport<nt current exampje is helicopter 
gunnery training. About one-fourth of the aviators are currently selected for this training 
and the current method of selection is random. An attempt was made to select on the 
basis cf flight grade-point average but this method appeared to training administrators, on 
a subjective bans, to produce poorer results. Now a large number of potential predictors 
are available to compare against final training grades in gunnery, and soon it will be 
possible to see how well the available data predict this performance. 

Even if this prediction is reasonably accurate, the job will be far from complete. 
While it has been shown that quality of training performance in military aviation is 
related to quality of operational performance (8), the correlation is far from perfect. 
Thus, simultaneously with the preparation for validation against training criteria, a 
PREDICT team has been engaged in the develo, ment of a measure of job performance 
(3, 9). This must be done simultaneously, because the requirement for the job— combat 
flying— may, hopefully, disappear soon. 



We have begun collecting validation data (in the fori:: of peer nominations, which 
seem the best initial measure of combat performance) from aviators returning from 
Vietnam. The number will grow at the rate of several hundred per month. 

A further problem, in addition to that of developing a job performance measure, is 
that a number of useful predictor measures are not now being generated in the Army 
system, and others require some evaluation before being used in the prediction system. 
One of these is a peer rating taken during training. Wahlberg (10) has reported on an 
ongoing study of peer ratings as predictors of aviator performance. 

The PREDICT research is moving in these various directions simultaneously because 
it is necessary, for good multivariate analysis, to have all potentially useful measures 
available on the same cases— and when the cases aie as mobile and as busy as Army 
aviators, rapid processing is essential. 

All these studies are aimed at a single goal: to have a comprehensive description of 
the subject’s characteristics available at all points in his training and operational develop- 
ment, and to have ail the valid predictors available at a point in time for use in 

manpower management decisions. Substantia], though perhaps not smooth, progress is 

being made toward that goal. 

Some of the results of the exploratory work into prediction of completion or 
attrition during initial rotary wing training is given for a relatively small sample of 715 

subjects. The main analyses will be based on subgroup jVs well in excess of 2,000. The 

data in this paper represent a partial evaluation of (a) FAST component scores, 

(b) averages of daily flight grades at two early points in training, and (c) scores on a 

measure of voluntary exposu^ to physical harm threat, the Background Activities 
Inventory (BAI). The criterion variable is the dichotomy of completion versus failure to 
complete the flight training program. 

Table 1 contains the bivariate correlations between these variables and completion of 
training for one subject classification: Warrant Officer Candidates who came into the 
program directly after basic training. This table reflects data that are available during the 

preflight stage of training, and, thus, there are no flight performance variables included in 

this analysis. Table 2 lists the cumulative shrunken multiple R s for variables selected by 
the Wherry^ Doolittle procedure for this point in training. Table 3 is a probability table 
based on the regression equations derived from this matrix. 

Table 1 

Variables Used in the Initial Matrix 
Warrant Officer Candidates 
Weeks 1 Through 4 



Variables 


Fail trpb) 


Mean 


1. Instrument Comprehension 


.137 


17.72 


2. Mechanical Information 


.114 


14.73 


3. Complex Movements 


.197 


24.13 


4. Visualization of Maneuvers 


.063 


18.51 


5. Helicopter Knowledge 


.1,41 


10.36 


6. Stick and Rudder Orientation 


.197 


25.99 


7. Aviation Information 


.134 


9.32 


8. Mechanical Functions 


.144 


15.35 


9. Background Activities Inventory (BAI) 


.070 


34,30 
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Table 2 



Cumulative B For Predictor Variables 
Warrant Officer Candidates 
Weeks 1, 2, 3, and 4 





Completed | Failed 


Total | 


N 


591 124 


715 


Shrunken Multiple R (Point Biserial) With Complete/Attrite 


Variable 


Cumulative R 


Complex Movements 




.197 


Stick and Rudder Orientation 


.250 


Mechanical Functions 




.265 


Aviation Information 




.273 


Background Activities Inventory (BAI) 


.277 (R b equivalent .412) 



Table 3 

Probability Table 

Enlistment Option Warrant Officer Candidates 
Weeks 1,2,3, and 4 





Percent 
of Total 


Number 

Complete 


Number 

Drop 


Approx. Odds 


Percent 

Graduated 


For 


Against 


1050 and Higher 


10 


68 


2 


34 


1 


97 


900 1049 


45 


289 


36 


8 


1 


89 


850-899 


18 


102 


27 


4 


1 


79 


800-849 


12 


65 


23 


3 


1 


74 


760799 


8 


40 


15 


2.67 


1 


73 


700749 


5 


23 


12 


2 


1 


68 


Under 700 


2 


4 


9 


1 


2 


31 


Totals 


100 


591 


124 


4.75 


1 


83 



The first eight variables shown in Table 1 are eight subtests of FAST, the primary 
selection instrument. The ninth is the aforementioned measure of voluntary exposure to 
threat of physical harm, the Background Activities Inventory. This first analysis, covering 
the first through fourth week of training, consists entirely of paper*and*pencil tests. As 
flying does not begin until the fifth week of training, the seventh arid eighth weeks of 
training, shown on Table 4, are the third and fourth weeks, respectively, of flying activity 
for the students. It is interesting to note the increase in *he multiple R (Table 5) and the 
changes in the probability table (Table 6), once training performance measures are 
included in the matrix. A somewhat arbitrary division has been made of flying grade 
Q ‘ it averages in the second matrix, into the first five graded flights and the sixth 
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through fourteenth graded flights. This is not entirely arbitrary, however, a student is 
rarely removed from the program for fly?- *: deficiencies until he has had five graded 
flights, while he may be removed on any gr ' .d flight thereafter. 

This is a somewhat inexact prediction when compared to explanation of all variance 
but, compared to what one person is able to do when faced with an individual subject’s 
scores on this same set of variables, it is good— good enough, in fact, to justify the 
construction of the more elaborate system whicu has been described in this paper. This 
will allow the construction of the best prediction system possible within the constraints 
of current multivariate analytic techniques, and within the limits of reliability of the 



Table 4 

Variables Used in Second Matrix 
Warrant Officer Candida^ 
Weeks 7 and 8 



Variables 


Pail (rpb) 


Mean 


1. Instrument Comprehension 


. 12 ? 


17.85 


2. Meci anicai Information 


.094 


14.83 


>. Complex Movements 


.170 


24.37 


4. Visualization of Maneuvers 


.043 


18.56 


5. Helicopter Knowledge 


.104 


10.46 


6. Stick and Rudder Orientation 


.223 


26.03 


1. Aviation Information 


.100 


9.46 


8. Mechanical Functions 


.151 


15.41 


9. Backyound Activities Inventory (BA1) 


.077 


34.34 


10. Giaded Flights 1*5 


.293 


2.62 


11. Graded Flights 6-14 


.446 


2.37 



Table 5 

Cumulative R For Predictor Variables 
Warrant Officer Candidates 
Weeks 7 and 8 



Completed 



Failed 



Totaf 



N 590 98 683 



Shrunken Multipla /7 (Point Biserlal) With Com pi elu/At trite 



Variable 

G.P.A. Graded Flights 6-14 
Stick and Rudder Orientation 
Mechanic?! Function? 
Mechanical Information 
Helicopter Knowledge 



Cumulative R 

.446 

.469 

.483 

.501 

.611 (R^ equivalent .797) 
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Table 6 



Probability Table 

Enlistment Option Warrant Officer Candidates 
Weeks 7 and 8 





Percent 
of Total 


Number 

Complete 


Number 

Drop 


Approx. Odds 


Percent 

Graduated 


For 


Against 


1200 or More 


04 


30 


0 


No Failures 


100 


800-1199 


56 


369 


19 


19 


1 


95 


551799 


27 


151 


33 


4.5 


1 


82 


400-550 


10 


35 


33 


Ever? 


51 


200-399 


2 


5 


10 


1 


2 


33 


199 or Less 


<1 


0 


3 


No Successes 


00 


Jo\ :> 


100 


590 


98 


6 


1 


86 



criteria available. It is worth noting that the B;~kground Activities Inventory (BAI), 
which makes a small, but significant, contribution to the multiple R in the matrix for the 
first four weeks of training, is now undergoing item analysis, and the revised version 
should give significant improvement of prediction. Further, it is worth attention that, 
while flight grades comprise most of the prediction at the seventh and eighth weeks, the 
FAST component scores substantially raise the multiple R. 

The particular FAST subtests selected differ from those selected in the first analysis. 
The equivalent biserial values at both stages exceed those reported by the' Navy in use in 
a system which has been highly satisfactory for naval aviation (H). We are confident that 
our first comprehensive matrices will give substantially improved prediction. It is also 
hoped that the PREDICT project will give as great a stability of prediction as the Havy 
multiple Rs which have held up extremely well from year to year. 
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